Display apparatus and position detecting method

ABSTRACT

An image generating unit of a display apparatus generates a determination image based on first and second light reception images obtained from light reception cells in a light-emission period and in a non-light-emission period, respectively. An image determining unit determines whether or not an image of an object to be detected is included in the determination image, stores data corresponding to the determination image as initial data into a storage when the object image is not included in the determination image, and stores the data corresponding to the determination image as detection data into the storage when the object image is included in the determination image. A position determining unit determines at least an object position based on an image represented by the detection data in the storage and an image represented by the initial data in the storage.

CROSS REFERENCES TO RELATED APPLICATIONS

The subject matter of application Ser. No. 12/390,925, is incorporatedherein by reference. The present application is a Continuation of U.S.Ser. No. 12/390,925, filed Feb. 23, 2009, which claims priority toJapanese Patent Application JP2008-059501, filed in the Japanese PatentOffice Mar. 10, 2008; the entire contents of which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a display apparatus having the functionof detecting a position and/or the like of an object (object to bedetected) which comes into contact with or comes close to a displaysurface, and a position detecting method for detecting the positionand/or the like of the object to be detected.

2. Description of the Related Art

Techniques for detecting a position and/or the like of an object whichcomes into contact with or comes close to a display surface of a displayapparatus are known. Among them, a typified and popularized technique isa display apparatus having a touch panel. There are touch panels ofvarious types, and a popularized one is a touch panel of a type ofdetecting capacitance. In this type, when a touch panel is touched witha finger, a change in surface charge of the panel is captured to detectthe position and/or the like of an object. By using such a touch panel,the user is possible to perform operation intuitively.

Recently, various techniques of enabling the position and/or the like ofan object to be detected without providing such a touch panel on thedisplay surface are proposed. For example, in an organic EL(Electro-Luminescence) display, a technique is proposed, ofintermittently performing display (light emission) in light emittingelements for image display disposed in the display surface, during theperiod in which the light emission stops, accumulating charges accordingto light reception in the light emitting elements themselves, andreading the stored charge amount. Similarly, for example, in a liquidcrystal display, a technique is proposed in which light receivingelements are disposed adjacent to display pixels and, during the periodin which display (light emission) stops, light is received by the lightreceiving elements. By using such a display apparatus, the positionand/or the like of an object is detectable on the basis of a capturedvideo image. Therefore, by using such a display apparatus, the positionand/or the like of an object is detectable with a simple configurationwithout providing a part such as a touch panel on the display surface.

However, at the time of capturing a video image and the like of anobject in such a display apparatus, the luminance of light received isinfluenced by the surrounding environment (brightness). Consequently, inthe case of providing a display apparatus whose environment easilychanges with the function of detecting the position and/or the like ofan object such as a portable electronic device in particular, the lightreceiving conditions in the dark environment and those in the lightenvironment are largely different from each other. It is thereforedifficult to detect the position and/or the like of an object underuniform light receiving conditions.

For example, Japanese Unexamined Patent Application Publication No.2007-25796 proposes the following measure for detecting the positionand/or the like of an object. That is, in a liquid crystal display, atthe time of displaying an image (moving image or still image) frame byframe, each of frame periods is halved. Then, a backlight is turned offin the first-half period to display no image, and the backlight isturned on and a display signal is simultaneously written to displaypixels in the latter-half period of each of the frame periods to displayan image. Further, a light reception signal is read from a lightreceiving element in the first-half period in which the backlight is offand the latter-half period in which the backlight is on. By using animage (first image) obtained by calculating the difference of the lightreception signals, the position and/or the like of an object isdetected. By the measure, the position and/or the like of an object isdetectable regardless of the environment (brightness).

However, in the first image, noise which occurs in the display apparatus(for example, noise caused by light from the backlight and variations insensitivity of the light receiving elements) is included. There is acase that it is difficult to detect the position and/or the like of anobject due to the influence of noise. To address the problem, theJapanese Patent Unexamined Patent Application Publication No. 2007-25796proposes another measure. That is, in a state where there is no externallight and there is no surface reflecting object (that is, in a statewhere there is no object which is in contact with or is close to thesurface), light reception signals are read from light receiving elementsin both of the case where a backlight is off and the case where thebacklight is on. An image (second image) obtained by calculating thedifference of the light reception signals is stored in advance in amemory. At the time of detecting the position and/or the like of anobject, each pixel data of the second image is subtracted from eachpixel data of the first image, thereby eliminating the noise componentfrom the first image.

SUMMARY OF THE INVENTION

In the technique of the Japanese Patent Unexamined Patent ApplicationPublication No. 2007-25796, however, it is necessary to store the secondimage in the memory in advance. Therefore, in the case where noiseoccurring in the display apparatus changes due to a change in theenvironment of the display apparatus, change with time, or the like,there is a possibility that the noise component may not be effectivelyremoved from the first image. Although the Japanese Patent UnexaminedPatent Application Publication No. 2007-25796 describes that the secondimage may be obtained at any time by an operation of the user of thedisplay apparatus, a concrete method of generating a state where thereis no external light and there is no surface reflecting object by theuser is not disclosed. It is also unrealistic for the user to generatesuch a state.

It is therefore desirable to provide a display apparatus and a positiondetecting method capable of detecting a position and/or the like of anobject which comes into contact with or comes close to the displaysurface of the display apparatus with high precision by automaticallyand effectively eliminating the noise component occurring in the displayapparatus.

A displaying apparatus according to an embodiment of the presentinvention includes: a panel part having a plurality of light emissioncells which display an image on a display surface and emit detectionlight, and a plurality of light reception cells which receive thedetection light incident from a side of the display surface; a storagestoring initial data and detection data; an image generating unitgenerating an image for determination on the basis of a first lightreception image and a second light reception image, in which the firstlight reception image is obtained from the plurality of light receptioncells in a first period in which the detection light is emitted from thedisplay surface, and the second light reception image is obtained fromthe plurality of light reception cells in a second period in which thedetection light is not emitted from the display surface; an imagedetermining unit determining whether or not an image of an object to bedetected is included in the image for determination, storing datacorresponding to the image for determination as the initial data intothe storage when the image of the object to be detected is determinednot to be included in the image for determination, and storing the datacorresponding to the image for determination as the detection data intothe storage when the image of the object to be detected is determined tobe included in the image for determination; and a position determiningunit determining at least a position of the object to be detected on thebasis of an image represented by the detection data stored in thestorage and an image represented by the initial data stored in thestorage.

A position detecting method according to an embodiment of the presentinvention detects at least a position of an object to be detected in adisplay apparatus including a panel part having a plurality of lightemission cells which display an image on a display surface and emitdetection light and a plurality of light reception cells which receivethe detection light incident from a side of the display surface, and astorage storing initial data and detection data. The method includes thesteps of: generating an image for determination on the basis of a firstlight reception image and a second light reception image, in which thefirst light reception image is obtained from the plurality of lightreception cells in a first period in which the detection light isemitted from the display surface, and the second light reception imageis obtained from the plurality of light reception cells in a secondperiod in which the detection light is not emitted from the displaysurface; determining whether or not an image of the object to bedetected is included in the image for determination, storing datacorresponding to the image for determination as the initial data intothe storage when the image of the object to be detected is determinednot to be included in the image for determination, and storing the datacorresponding to the image for determination as the detection data intothe storage when the image of the object to be detected is determined tobe included in the image for determination; and determining at least theposition of the object to be detected on the basis of an imagerepresented by the detection data stored in the storage and an imagerepresented by the initial data stored in the storage.

In the display apparatus and the position detecting method according tothe embodiments of the present invention, when emission of the detectionlight and non-emission of the detection light are sequentiallyperformed, the data corresponding to the image for determinationgenerated on the basis of the first light reception image obtained inthe light emission period of the detection light and the second lightreception image obtained in the non-light emission period of thedetection light is used for determining at least the position of theobject to be detected. As a result, the influence of external light atthe time of detecting the position and/or the like of the object iseliminated. Therefore, even in the case where surrounding environment(brightness) changes, regardless of the surrounding environment, theposition and/or the like of the object which comes into contact with orcomes close to the display surface of a display apparatus is easilydetected. In addition, the data corresponding to the image fordetermination determined as an image which does not include the image ofthe object to be detected is used as the initial data. Consequently, theinfluence of noise occurring in the display apparatus at the time ofdetecting the position and/or the like of the object is eliminated.Therefore, even in the case where noise occurring in the displayapparatus changes due to a change in the environment of the displayapparatus, a change with time, or the like, a noise component generatedin the display apparatus is eliminated automatically and effectively,without requesting any operation from the user of the display apparatus.

According to the display apparatus and the position detecting method ofthe embodiments of the present invention, by automatically andeffectively eliminating the noise component occurring in the displayapparatus, the position and/or the like of the object which comes intocontact with or comes close to the display surface of the displayapparatus is detected with high precision.

Other and further objects, features and advantages of the invention willappear more fully from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the general configuration of a displayapparatus according to an embodiment of the present invention.

FIG. 2 is a plan view schematically showing an example of theconfiguration of light emission cells and light reception cells in FIG.1.

FIG. 3 is a cross section schematically showing an example of theconfiguration of light emitting elements in FIG. 1.

FIG. 4 is a circuit diagram schematically showing an example of theconfiguration of each pixel in FIG. 1.

FIG. 5 is a timing chart showing an example of a process of detecting anobject to be detected by line sequential operation.

FIG. 6 is a timing chart showing another example of a process ofdetecting an object to be detected by line sequential operation.

FIG. 7 is a block diagram schematically showing an example of theconfiguration of a position detector in FIG. 1.

FIG. 8A is a side view schematically showing a state where externallight is strong, and FIG. 8B is a waveform chart showing an example ofreception light output voltage when external light is strong.

FIG. 9A is a side view schematically showing a state where externallight is weak, and FIG. 9B is a waveform chart showing an example ofreception light output voltage when external light is weak.

FIGS. 10A to 10C are schematic diagrams for explaining a method ofdetermining whether an image of an object to be detected is included inan image for determination or not.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

One embodiment of the present invention will be described in detailhereinbelow with reference to the drawings.

FIG. 1 shows a general configuration of a display apparatus according toan embodiment of the present invention. The display apparatus has adisplay unit 1, a display signal generator 21, a display signalholder/controller 22, a display signal driver 23, a light emissionscanner 24, a light reception scanner 31, a light reception signalreceiver 32, a light reception signal holder 33, and a position detector34. The display apparatus is possible to display an image and receivelight. The display apparatus displays an image based on image data onthe display unit 1, and detects a position and/or the like of an object(object to be detected) which comes into contact or comes close to thedisplay unit 1. A position detecting method according to an embodimentof the invention is embodied by the display apparatus, so that it willbe described in conjunction therewith.

The display unit 1 includes an LCD (Liquid Crystal Display) in which aplurality of pixels 11 are disposed in a matrix on the entire surface ofthe display unit 1. As will be described later, the display unit 1displays an image of a predetermined figure, character, or the likewhile performing line-sequential operation.

FIG. 2 is a plan view showing an example of the configuration of eachpixel 11. Each pixel 11 is constructed by light emission cells 12 foremitting visible light toward a display surface 10 (refer to FIG. 3) ofthe display unit 1, and a light reception cell 13 for receiving thevisible light incident from the display surface 10 side. The lightemission cells 12 have a red light emission cell 12R for emitting redlight, a green light emission cell 12G for emitting green light, and ablue light emission cell 12B for emitting blue light. The red lightemission cell 12R has a red light emitting element 14R as a part foremitting red light and a TFT (Thin Film Transistor) circuit part 15R fordriving the red light emitting element 14R. Similarly, the green lightemission cell 12G has a green light emitting element 14G as a part foremitting green light and a TFT circuit part 15G for driving the greenlight emitting element 14G. The blue light emission cell 12B has a bluelight emitting element 14B as a part for emitting blue light and a TFTcircuit part 15B for driving the blue light emitting element 14B. On theother hand, the light reception cell 13 has a light receiving element 16as a part for receiving visible light and a light reception sensorcircuit part 17 for driving the light receiving element 16. The lightreceiving element 16 includes, for example, a photodiode. The details ofthe connection relations between the red light emitting element 14R, thegreen light emitting element 14G, and the blue light emitting element14B and the TFT circuit parts 15 and the connection relations betweenthe TFT circuit parts 15 and the display signal driver 23, the lightemission scanner 24, the light reception scanner 31, and the lightreception signal receiver 32 will be described later (FIG. 4).

FIG. 3 is a cross section taken along line A-A of FIG. 2 and shows anexample of a sectional configuration of the display unit 1. The displayunit 1 has, on a light source 100, a panel part 110 having a stackstructure constructing the light emitting elements (red light emittingelement 14R, green light emitting element 14G, and blue light emittingelement 14B). The panel part 110 is a so-called liquid crystal displaypanel and concretely has, from the light source 100 side, a polarizer101A, a transparent substrate 102A, a circuit part 103, an insulatinglayer 104, a transparent pixel electrode 105A, a liquid crystal layer106, a transparent electrode 105B, a color filter 107, a black matrix109, a transparent substrate 102B, and a polarizer 101B. The lightemitting elements (red light emitting element 14R, green light emittingelement 14G, and blue light emitting element 14B) are provided in aliquid crystal display panel in which the liquid crystal layer 106 isprovided between the transparent substrates 102A and 102B which faceeach other.

The light source 100 is a backlight for emitting light in the visiblelight range toward the liquid crystal elements. The transparentsubstrates 102A and 102B are made of, for example, a glass material. Thetransparent substrates 102A and 102B may be made of a transparentplastic material or the like instead of the glass material.

The circuit part 103 is a part corresponding to the TFT circuit part 15and the light reception sensor circuit part 17 shown in FIG. 2 and iselectrically connected to the transparent pixel electrodes 105A. Thetransparent pixel electrodes 105A are disposed in the light emittingelements and are made of a transparent material such as ITO (Indium TinOxide). On the other hand, the transparent electrode 105B is a commonelectrode facing the transparent electrode 105A and is made of atransparent material such as ITO like the transparent electrode 105A.The insulating layer 104 is formed between the circuit parts 103. Withsuch a configuration, a voltage according to display data is appliedacross the transparent electrodes 105A and 105B, and backlight light Lofrom the light source 100 passes through the liquid crystal layer 106 oris interrupted.

The color filter 107 is disposed in an area corresponding to the lightemission cells 12 (red light emission cell 12R, green light emissioncell 12G, and blue light emission cell 12B) and selectively transmitslight in the wavelength range corresponding to the light emission colorof itself in the backlight light Lo passed through the liquid crystallayer 106. The black matrix 109 is disposed between the color filters107 and interrupts the backlight light Lo from the light source 100 sothat the backlight light Lo does not go to the display surface 10 side.

FIG. 4 shows an example of the circuit configuration in each of thepixels 11. Each pixel 11 has, as described above, the light emissioncell including the red light emission cell 12R, green light emissioncell 12G, and blue light emission cell 12B, and the light reception cell13. To the light emission cell 12, a display data supply line DWconnected to the display signal driver 23 and a light emission gate lineGW connected to the light emission scanner 24 are connected. Concretely,a display data supply line DWr and the light emission gate line GW areconnected to the red light emission cell 12R. A display data supply lineDWg and the light emission gate line GW are connected to the green lightemission cell 12G. A display data supply line DWb and the light emissiongate line GW are connected to the blue light emission cell 12B. On theother hand, to the light reception cell 13, a light reception gate lineGR connected to the light reception scanner 31, a light reception resetline RR, and a data read line DR connected to the light reception signalreceiver 32 are connected.

The red light emission cell 12R has the red light emitting element 14Rand the TFT circuit part 15R including a light emitting elementselection switch SW1R for driving the red light emitting element 14R.One end of the light emitting element selection switch SW1R is connectedto the display data supply line DWr, the other end is connected to oneend of the red light emitting element 14R (concretely, the transparentpixel electrode 105A) and, further, the other end of the red lightemitting element 14R (concretely, the transparent electrode 105B) isgrounded. Similarly, the green light emission cell 12G has the greenlight emitting element 14G and the TFT circuit part 15G including alight emitting element selection switch SWIG for driving the green lightemitting element 14G. One end of the light emitting element selectionswitch SWIG is connected to the display data supply line DWg, the otherend is connected to one end of the green light emitting element 14G and,further, the other end of the green light emitting element 14G isgrounded. The blue light emission cell 12B has the blue light emittingelement 14B and the TFT circuit part 15B including a light emittingelement selection switch SW1B for driving the blue light emittingelement 14B. One end of the light emitting element selection switch SW1Bis connected to the display data supply line DWb, the other end isconnected to one end of the blue light emitting element 14B and,further, the other end of the blue light emitting element 14B isgrounded. The on/off operation of each of the light emitting elementselection switches SW1R, SW1G, and SW1B is controlled by the lightemission gate line GW. Each of the light emitting element selectionswitches SW1R, SW1G, and SW1B includes a switch element such as a TFT.

On the other hand, the light reception cell 13 has the light receivingelement 16 (the photodiode in the example of FIG. 4), and lightreceiving element selection switches SW2 and SW3, a buffer amplifierAMP, and a capacitor C provided in the light emitting element circuitpart 17. One end of the light receiving element 16 is connected to apower source line VDD and the other end is connected to the inputterminal of the buffer amplifier AMP. An output terminal of the bufferamplifier AMP is connected to one end of the light receiving elementselection switch SW2, and the other end of the light receiving elementselection switch SW2 is connected to the data read line DR. Further, aninput terminal of the buffer amplifier AMP is connected to one end ofthe light receiving element selection switch SW3 and one end of thecapacitor C. The other end of the light receiving element selectionswitch SW3 and that of the capacitor C are grounded. The on/offoperation of the light receiving element selection switch SW2 iscontrolled by the light reception gate line GR. The on/off operation ofthe light receiving element selection switch SW3 is controlled by thelight reception reset line RR. Each of the light receiving elementselection switches SW2 and SW3 includes a switch element such as a TFT.

Next, the configuration other than the display unit 1 in the displayapparatus (the display signal generator 21, display signalholder/controller 22, display signal driver 23, light emission scanner24, light reception scanner 31, light reception signal receiver 32,light reception signal holder 33, and position detector 34) will bedescribed.

The display signal generator 21 generates a display signal 21A to bedisplayed on the display unit 1 for each frame a field on the basis ofimage data supplied from a not-shown CPU (Central Processing Unit) orthe like. The display signal 21A generated as described above is outputto the display signal holder/controller 22.

The display signal holder/controller 22 holds the display signal 21Aoutput from the display signal generator 21 by storing the displaysignal 21A screen by screen (every display of a field) into a fieldmemory including, for example, an SRAM (Static Random Access Memory).The display signal holder/controller 22 also plays the role ofperforming control so that the display signal driver 23 and the lightemission scanner 24 for driving the light emission cells 12, the lightreception scanner 31 for driving the light reception cells 13, and thelight source 100 (which will be described later) of the display unit 1operate interlockingly. Concretely, the display signal holder/controller22 outputs a light emission timing control signal 22A to the lightemission scanner 24, outputs a light reception timing control signal 22Bto the light reception scanner 31, outputs a display signal 22C of onehorizontal line based on the display signal 21A of one screen held inthe field memory to the display signal driver 23, and outputs a lightsource control signal 22D for turning on/off the light source 100 to thelight source 100 of the display unit 1. More specifically, for example,as shown in FIGS. 5 and 6, at the time of displaying an image (movingimage or still image) frame by frame, the display signalholder/controller 22 halves each of frame periods T. Then, the displaysignal holder/controller 22 outputs a signal for turning on the lightsource 100 as the light source control signal 22D to turn on the entirelight source 100 in the first-half period T1 or in the latter-halfperiod T2. During the light-on period, the display signalholder/controller 22 outputs the light emission timing control signal22A, the light reception timing control signal 22B, and the displaysignal 22C to make the light emission cells 12 and the light receptioncells 13 synchronously driven in the direction of the arrow X (seeFIG. 1) every horizontal line (line-sequential drive). In addition,during a period different from the light-on period which is either thefirst-half or latter half period in each frame period, the displaysignal holder/controller 22 outputs a signal for turning off the lightsource 100 as a light source control signal 22D to turn off the entirelight source 100 and, during the turn-off period, outputs the lightreception timing control signal 22B to make the light reception cells 13driven in the direction of the arrow X every horizontal line(line-sequential drive) every horizontal line. The light reception cells13 may not be continuously driven in one frame cycle, and the lightreception cells 13 may be intermittently driven in plural frame cyclesas necessary.

The display signal driver 23 supplies display data to the light emissioncell 12 to be driven in accordance with the display signal 22C of onehorizontal line output from the display signal holder/controller 22.Concretely, the display signal driver 23 supplies a voltage 23Acorresponding to the display data to the pixel 11 selected by the lightemission scanner 24 via the data supply line DW connected to the pixels11 of the display unit 1.

The light emission scanner 24 selects the light emission cell 12 to bedriven in accordance with the light emission timing control signal 22Boutput from the display signal holder/controller 22. Concretely, thelight emission scanner 24 supplies a light emission selection signal 24Ato the visible light emission cell 12 to be driven via the lightemission gate line GW connected to the pixels 11 of the display unit 1to control the light emitting element selection switches SW1R, SWIG, andSW1B. In such a manner, when the voltage to turn on the light emittingelement selection switches SW1R, SW1G, and SW1B of the certain pixel 11is applied to the light emitting element selection switches SW1R, SW1G,and SW1B by the light emission selection signal 24A, an operation ofemitting light having luminance corresponding to the voltage 23Asupplied from the display signal driver 23 is performed in that pixel11. When the light emission scanner 24 and the display signal driver 23perform the line-sequential operation interlockingly, an imagecorresponding to arbitrary display data is displayed on the display unit1.

The light reception scanner 31 selects the light reception cell 13 to bedriven in accordance with the light reception timing control signal 22Boutput from the display signal holder/controller 22. Concretely, thelight reception scanner 31 supplies a light reception selection signal31A to the light reception cell 13 to be driven via the light receptiongate line GR connected to the pixels 11 of the display unit 1 to controlthe light receiving element selection switch SW2, and supplies a resetsignal 31B to the light reception cell 13 to be driven via the lightreception reset line RR connected to the pixels 11 in the display unit 1to control the light receiving element selection switch SW3. That is,when the voltage to turn on the light receiving element selection switchSW3 in the certain pixel 11 is applied to the light receiving elementselection switch SW3 by the reset signal 31B, charges accumulated in thecapacitor C in that pixel 11 are reset. When the voltage to turn on thelight receiving element selection switch SW2 in the certain pixel 11 isapplied to the light receiving element selection switch SW2 by the lightreception selection signal 31A, charges accumulated in the capacitor Cin correspondence with the light reception amount in the light receivingelement 16 in that pixel 11 are output as a light reception signal 1A tothe light reception signal receiver 32 via the buffer amplifier AMP andthe data read line DR. In such a manner, visible light is received bythe light reception cell 13.

The light reception scanner 31 also plays the role of outputting a lightreception block control signal 31C to the light reception signalreceiver 32 and the light reception signal holder 33 to control theoperation of a part contributing to the light receiving operation.

The light reception signal receiver 32 obtains the light receptionsignals 1A of one horizontal line output from the light reception cells13 in accordance with the light reception block control signal 31Coutput from the light reception scanner 31. The light reception signals1A of one horizontal line thus obtained are output to the lightreception signal holder 33.

The light reception signal holder 33 reconstructs a light receptionsignal 32A output from the light reception signal receiver 32 to a lightreception signal 33A (light reception image) every screen (every displayof one field) in accordance with the light reception block controlsignal 31C output from the light reception scanner 31, and stores andholds the light reception signal 33A in a field memory (not shown) suchas an SRAM. The light reception signal 33A thus stored in the fieldmemory is output to the position detector 34. The light reception signalholder 33 may have a storing device other than the memory and may hold,for example, the light reception signal 33A as analog data.

The position detector 34 specifies the position and/or the like of anobject (object to be detected) which comes into contact with or comesclose to the display surface 10 by performing a predetermined signalprocess on the light reception signal 33A output from the lightreception signal holder 33. In the case where the light reception signalholder 33 holds the light reception signal 33A as analog data, theposition detector 34 may perform an analog-to-digital conversion (A/Dconversion) and, after that, execute the signal process.

FIG. 7 shows functional blocks of the position detector 34. The positiondetector 34 has, as shown in FIG. 7, a reflection detection processor41, a shade detection processor 42, a synthesis processor 43, aproximity detection processor 44, and a storage 45. The reflectiondetection processor 41, the shade detection processor 42, or thesynthesis processor 43 in the present embodiment corresponds to one ofexamples of the “image generating unit” of the present invention.

According to the light source control signal 22D output from the displaysignal holder/controller 22, the reflection detection processor 41classifies the light reception signals 33A output from the lightreception signal holder 33 to a light-on light reception image (lightreception image when an image is displayed) obtained in a light-onperiod (image display period) and a light-off light reception image(light reception image when no image is displayed) obtained in alight-off period (no-image display period). The reflection detectionprocessor 41 obtains the difference between the light-on light receptionimage and the light-off light reception image (for example, bysubtracting data of the light-off light reception image from thelight-on light reception image), generates a difference image 41A, andoutputs the difference image 41A to the synthesis processor 43. Thelight-on light reception image is a light reception image obtained in aperiod in which detection light of the light reception cell 13 is outputfrom the display surface 10. The light-off light reception image is alight reception image obtained in a period in which the detection lightof the light reception cell 13 is not output from the display surface10.

Since each of the light-on light reception image and the light-off lightreception image includes a component of external light, by taking thedifference between the light-on light reception image and the light-offlight reception image, the component of the external light iseliminated. As a result, an image which does not depend on thesurrounding environment (brightness) (difference image 41A) isobtainable.

For example, when incident external light L2 is strong as shown in FIG.8A, a light reception output voltage Von1 in a state where the lightsource 100 is on has a voltage value Va corresponding to brightness ofthe external light L2 in the area other than the place touched with afinger. The light reception output voltage Von1 drops to a voltage valueVb corresponding to reflectance of light from the light source 100 atthe surface of the touched finger in the place touched with the finger.On the contrary, although a light reception output voltage Voff1 in astate where the light source 100 is turned off similarly has the voltagevalue Va corresponding to brightness of the external light L2 in thearea other than the place touched with the finger, in the place touchedwith a finger, the external light L2 is interrupted, and the lightreception output voltage Voff1 has a voltage value Vc at a very lowlevel.

In a state where the incident external light L2 is weak (or hardlyexists) as shown in FIG. 9A, a reception light output voltage Von2 in astate where the light source 100 is on has a value close to the voltagevalue Vc of a very low level for the reason that there is no externallight. In the place touched with the finger, the reception light outputvoltage Von2 rises to the voltage value Vb corresponding to reflectanceof light from the light source 100 on the surface of the touched finger.On the contrary, a reception light output voltage Voff2 in the statewhere the light source 100 is turned off is unchanged and has thevoltage value Vc of the very low level in both of the place touched withthe finger and the other area.

As understood by comparison between FIGS. 8B and 9B, in a place wherethe display surface 10 of the panel part 110 is not touched, the lightreception output voltage in the case where there is the external lightL2 and that in the case where the external light L2 is weak (or hardlyexists) are largely different from each other. However, the lightreception output voltage in the case where the light source 100 is onand that in the case where the light source 100 is off are unchanged andare the same. Therefore, by detecting the difference between thelight-on light reception image and the light-off light reception image,an image from which the component of the external light L2 is eliminated(difference image 41A) is obtainable. In the place where the fingertouches, irrespective of the intensity of the external light L2, adifference (increase of voltage) between the voltage Vb when the lightsource 100 is on and the voltage Vc when the light source 100 is off isalmost the same. Consequently, the difference image 41A in the casewhere there is the external light L2 and that in the case where theexternal light L2 is weak (or hardly exists) are almost the same.Therefore, it is unlikely that the profile of the difference image 41Aobtained fluctuates due to the influence of the external light L2. Asdescribed above, in the embodiment, the image which does not depend onthe surrounding environment (brightness) is obtainable.

The shade detection processor 42 generates a shade image 42A byperforming the procedure described below and outputs the shade image 42Ato the synthesis processor 43.

First, the shade detection processor 42 generates a reversed image and amoving-averaged image of the light-off light reception image (lightreception signal 33A) obtained in the light-off period. The reversedimage is obtained by, for example, reversing the brightness/darkness ofthe light-off light reception image. The moving-averaged image isobtained by, for example, performing an averaging process which obtainsan average value of plural pixel data of a pixel region including atarget pixel and its peripheral pixels in the light-off light receptionimage, using image data obtained by the averaging process for anaveraging process on a pixel region of a next target pixel and itsperipheral pixels in the light-off light reception image, sequentiallyshifting the target pixel, and performing the averaging process on theentire light-off light reception image.

Desirably, the size of the pixel region on which the averaging processis performed (the number of pixels in vertical direction and horizontaldirection) is set on the basis of an expected size of an image of theobject to be detected. For example, the size of the pixel region onwhich the averaging process is performed is set to the size almost thesame as that of the object to be detected. In this case, when an imagehaving a size larger than that of the pixel region to be subject to theaveraging process is included in the light-off light reception image,the contour of the image is blurred by the averaging process. Therefore,for example, when the object to be detected is a fingertip, by settingthe size the pixel region to be subject to the averaging process almostthe same as that of an image of the fingertip, the contour of an imagelarger than the image of the fingertip is blurred, and at the time ofgenerating the difference image 41A which will be described later, animage (for example, an image of a fist) larger than the image of thefingertip is eliminated. As a result, the fist is prevented from beingerroneously detected.

In the case where the averaging process is performed by using theabove-described method, the peripheral region of the light-off lightreception image lies out of the target pixels. In that case, it ispreferable that some interpolating process be performed on the pixeldata in the peripheral region. For example, pixel data of pixels at theoutermost periphery of the region subjected to the averaging process inthe light-off light reception image may be copied as it is as pixel dataof pixels on the outside of the pixels.

Next, the shade detection processor 42 calculates a predeterminedthreshold for use in a subsequent process from the moving-averagedimage. Concretely, a threshold is calculated on the basis of pixel dataof a pixel having highest brightness (largest pixel data) in themoving-averaged image and pixel data of a pixel having lowest brightness(smallest pixel data) in the light-off light reception image prior tothe averaging process (for example, by averaging the pixel data). To thepixel data of a pixel having highest brightness (largest pixel data), onassumption that an object to be detected is normally not disposedsimultaneously at the four corners of the display surface 10, theaverage value of pixel data of pixels at the four corners may beassigned.

Then, the shade detection processor 42 generates a reversed image of themoving-averaged image. The reversed image is obtained by, for example,reversing the brightness/darkness of the moving-averaged image.Subsequently, the shade detection processor 42 obtains the differencebetween the reversed image of the light-off light reception image andthe reversed image of the moving-averaged image (for example, bysubtracting the data of the reversed image of the moving-averaged imagefrom the data of the reversed image of the light-off light receptionimage), and the difference image is generated. Then, by subtracting thethreshold which is obtained in advance from the pixel data of thedifference image, the shade image 42A is generated.

By subtracting the threshold from the pixel data of the differenceimage, the value of the pixel data corresponding to the image of anobject other than the object to be detected included in the differenceimage is decreased. As a result, erroneous detection of the object otherthan the object to be detected is avoided.

The synthesis processor 43 combines the difference image 41A output fromthe reflection detection processor 41 and the shade image 42A outputfrom the shade detection processor 42, thereby generating a syntheticimage 43A, and outputs the synthetic image 43A to the proximitydetection processor 44.

Preferably, the synthesis of the difference image 41A and the shadeimage 42A is performed by adding data obtained by multiplying each ofthe pixel data of the difference image 41A with a predeterminedcoefficient α (0≦α≦1) and data obtained by multiplying each of the pixeldata of the shade image 42A with a predetermined coefficient (1−α).Obviously, the synthetic image 43A may be also generated by anothermethod.

As necessary, in place of the output (synthetic image 43A) of thesynthesis processor 43, the output (difference image 41A) of thereflection detection processor 41 or the output (shade image 42A) of theshade detection processor 42 may be input to the proximity detectionprocessor 44. For example, when a semi-transmissive liquid crystaldevice for the panel part 110 is used as in the embodiment, in the casesuch that the light source 100 is always in the off state or a blackimage is displayed on the panel part 110, it is preferable to input theoutput of the shade detection processor 42 (shade image 42A) to theproximity detection processor 44.

The proximity detection processor 44 has a determination processor 51and an object specifying processor 52. The determination processor 51 ofthe present embodiment corresponds to one of examples of “imagedetermining unit” of the present invention. The object specifyingprocessor 52 of the embodiment corresponds to one of examples of“position determining unit” of the present invention.

The determination processor 51 determines whether or not an image of anobject to be detected is included in an image generated on the basis ofthe outputs of the light reception signal holder 33 (light-on lightreception image and light-off light reception image) and updates initialdata 45A and detection data 45B in the storage 45 in accordance with thedetermination result. In the case where an output of the synthesisprocessor 43 is input to the proximity detection processor 44, an image(image for determination) generated on the basis of the output of thelight reception signal holder 33 corresponds to the synthetic image 43A.In the case where an output of the reflection detection processor 41 isinput to the proximity detection processor 44, the image fordetermination corresponds to the difference image 41A. In the case wherean output of the shade detection processor 42 is input to the proximitydetection processor 44, the image for determination corresponds to theshade image 42A.

Whether an image of the object to be detected is included in the imagefor determination or not is determined by, for example, as shown inFIGS. 10A to 10C, detecting whether or not a pixel region R having avalue exceeding a predetermined threshold TH exists in the pixel data ofthe image for determination, and by, in the case where the pixel regionR is detected, comparing the size of the pixel region R with expectedsize of an image of an object to be detected. For example, in the casewhere the horizontal and vertical size of a target region R is equal toor larger than an expected size (M×N (where M and N are numbers ofpixels)) of an image of an object to be detected, the determinationprocessor 51 determines that the image of the object to be detected isincluded in the image for determination. In the case where thehorizontal and vertical size of the target region R is smaller than theexpected size (M×N (where M and N are numbers of pixels)) of the imageof the object to be detected, the determination processor 51 determinesthat the image of the object to be detected is not included in the imagefor determination. In such a manner, the determination processor 51determines whether an image of an object to be detected is included inan image for determination or not on the basis of a simple rule.

When the determination processor 51 determines that the image of theobject to be detected is not included in the image for determination,the image for determination is stored as the initial data 45A in thestorage 45. When the determination processor 51 determines that theimage of the object to be detected is included in the image fordetermination, the image for determination is stored as the detectiondata 45B in the storage 45. The image for determination is stored in thestorage 45 and, thereafter, a determination process end signal 51A isoutput to the object specifying processor 52.

In the case where default data is pre-stored as the initial data 45A inthe storage 45 or in the case where an image for determination generatedon the basis of outputs (light-on light reception image and light-offlight reception image) of the light reception signal holder 33 in a pastframe period T for example is already stored as the initial data 45A bythe process of the determination processor 51, the image fordetermination may be overwritten as the latest initial data 45A on thestorage 45. Alternatively, the storage 45 may be overwritten with asynthetic image, as the latest initial data 45A, obtained by combiningthe image already stored as the initial data 45A and the image fordetermination. In addition, at the time of storing the image fordetermination as the detection data 45B in the storage 45, the image fordetermination or an image obtained by performing some process on theimage for determination may be stored as the detection data 45B in thestorage 45.

Preferably, the synthesis between the image already stored as theinitial data 45A and the image for determination newly stored as theinitial data 45A is performed by adding data obtained by multiplyingeach of the pixel data of the image already stored as the initial data45A with a predetermined coefficient β (0<β<1) and data obtained bymultiplying each of the pixel data of the image for determination newlystored as the initial data 45A with a predetermined coefficient (1−β).Obviously, the synthetic image may be also generated by another method.

When the determination process end signal 51A output from thedetermination processor 51 is received, the object specifying processor52 generates an image for analysis by subtracting the image stored asthe initial data 45A in the storage 45 from the image stored as thedetection data 45B in the storage 45. The object specifying processor 52derives the position information 44A of the object to be detected byusing the image for analysis (for example, by binarizing the image foranalysis), and outputs the position information 44A to, for example, aCPU (not shown).

By subtracting the image stored as the initial data 45A in the storage45 from the image stored as the detection data 45B in the storage 45,only the value of the pixel data corresponding to the image of theobject to be detected, included in the image stored as the detectiondata 45B in the storage 45 is extracted, so that the positioninformation 44A is easily derived.

Next, an example of the operation of the display apparatus of theembodiment will be described in detail.

In the display apparatus, the drive signals for display (the voltage 23Aand the light emission selection signal 24A) are generated by thedisplay signal driver 23 and the light emission scanner 24 on the basisof the display data supplied from a not-shown CPU or the like. By thedrive signals, the line-sequential display driving is performed in thedisplay unit 1 and an image is displayed. In addition, the lightreception drive signal (the light reception selection signal 31A) isgenerated from the light reception scanner 31. By the drive signal, theline-sequential light reception driving is performed in the display unit1, and an image is captured. At this time, the light source 100 isdriven by the display signal holder/controller 22, and the on/offoperation synchronized with the display unit 1 is performed.

Concretely, for example, as shown in FIG. 5, at the time of displayingan image frame by frame, each of frame periods T is halved. In afirst-half period T1, a signal for turning on the light source 100 isoutput as the light source control signal 22D to turn on the lightsource 100. In the light-on period, the display drive signals (thevoltage 23A and the light emission selection signal 24A) and the lightreception drive signal (the light reception selection signal 31A) areoutput. The light emission cells 12 and the light reception cells 13 aredriven synchronously every horizontal line, for example, in thedirection of the arrow X (line-sequential driving) (refer to FIG. 1).Thereby, images in the frame period are displayed, and the lightreception signal 33A in the first-half period T1 is obtained as thelight-on light reception image (the light reception image when an imageis displayed). Further, in a latter-half period T2 of each of the frameperiods, a signal for turning off the light source 100 is output as thelight source control signal 22D to turn off the entire light source 100.In the light-off period, the light reception drive signal (lightreception selection signal 31A) is output to drive the light receptioncells 13 every horizontal line in the direction of, for example, thearrow X (line-sequential driving), and the light reception signal 33A inthe latter-half period T2 is obtained as the light-off light receptionimage (light reception image when no image is displayed). In the casewhere the light reception cells 13 are driven intermittently in pluralframe cycles, the light reception signal 33A is obtained as the lightreception image in the light-off period only for the frame periods inwhich the light reception cells 13 are driven.

In the position detector 34, signal process is performed on the basis ofthe light reception signal 33A output from the light reception signalholder 33, and the position and/or the like of an object (object to bedetected) which comes into contact with or comes close to the displaysurface 10 is specified.

[Image Generating Step]

Concretely, first, in the reflection detection processor 41, the lightreception signals 33A output from the light reception signal holder 33are classified to a light-on light reception image and a light-off lightreception image in accordance with the light source control signal 22Doutput from the display signal holder/controller 22, and the differenceimage 41A is generated from the difference between the light-on lightreception image and the light-off light reception image. In addition, inthe shade detection processor 42, a reversed image and a moving-averagedimage are generated from the light-off light reception image, and areversed image and a threshold are generated from the moving-averagedimage. Further, in the shade detection processor 42, a difference imageis generated from the difference between the reversed image of thelight-off light reception image and the reversed image of themoving-averaged image, and the shade image 42A is generated bysubtracting the threshold preliminarily determined from the pixel dataof the difference image. Subsequently, in the synthesis processor 43,the synthesis image 43A is generated by combining the difference image41A and the shade image 42A according to a predetermined rule.

[Image Determining Step]

Next, in the determination processor 51, whether or not an image of anobject to be detected is included in the image for determination (thesynthesis image 43A, the difference image 41A or the shade image 42A)generated on the basis of the outputs (the light-on light receptionimage and the light-off light reception image) of the light receptionsignal holder 33 is determined on the basis of the simple rule. Further,in the determination processor 51, when it is determined that the imageof the object to be detected is not included in the above-describedimage, the image is stored as the initial data 45A in the storage 45.When it is determined that the image of the object to be detected isincluded in the above-described image, the image is stored as thedetection data 45B in the storage 45.

In the case where default data is pre-stored as the initial data 45A inthe storage 45 or in the case where an image for determination generatedon the basis of outputs (light-on light reception image and light-offlight reception image) of the light reception signal holder 33 in a pastframe period T for example is already stored as the initial data 45A bythe process of the determination processor 51, the image fordetermination may be overwritten as the latest initial data 45A on thestorage 45. Alternatively, the storage 45 may be overwritten with asynthetic image, as the latest initial data 45A, obtained by combiningthe image already stored as the initial data 45A and the image fordetermination in accordance with a predetermined rule.

[Position Deriving Step]

Next, in the object specifying processor 52, an image for analysis isgenerated by subtracting the image stored as the initial data 45A in thestorage 45 from the image stored as the detection data 45B in thestorage 45. By using the analysis image, the position information 44A ofthe object to be detected is derived and is output to, for example, aCPU (not shown). In such a manner, in the embodiment, the positionand/or the like of the object (object to be detected) which comes intocontact with or comes close to the display surface 10 is specified.

As described above, in the embodiment, when turn-on of the light source100 (display of an image) and turn-off of the light source 100 (displayof no image) are sequentially performed in a one-frame period, alight-on light reception image (light reception image when an image isdisplayed) is obtained in the light-on period (image display period),and a light-off light reception image (light reception image when noimage is displayed) is obtained in the light-off period (no-imagedisplay period). Images obtained by performing a predetermined signalprocess on the light-on light reception image and the light-off lightreception image are used for deriving the position of the object to bedetected. Therefore, the influence of external light at the time ofdetecting the position and/or the like of an object is eliminated.Consequently, even when the surrounding environment (brightness)changes, irrespective of the surrounding environment, the positionand/or the like of an object which comes into contact with or comesclose to the display surface of the display apparatus is easilydetected.

Since the image obtained by performing a predetermined process on theimage for determination is used as the initial data 45A, the influenceof noise occurring in the display apparatus at the time of detecting theposition and/or the like of an object is eliminated. Consequently, evenin the case where noise occurring in the display apparatus is changed bychange in the surrounding environment of the display apparatus, changewith time, or the like, the noise component generated in the displayapparatus is effectively eliminated. Therefore, the position and/or thelike of an object which comes into contact with or comes close to thedisplay surface of the display apparatus is detected with highprecision. In addition, since it is unnecessary to store data foreliminating noise generated in the display apparatus prepared at thetime of shipment from a factory as the initial data 45A in the storage45, the man-hour at the time of shipment from a factory is reduced.

In the embodiment, whether an image of an object to be detected isincluded in the image for determination or not is determined on thebasis of the simple rule as described above. Therefore, the process timeand load required to generate data to be stored in the storage 45 as theinitial data 45A is sufficiently reduced, and decrease in the responsespeed of position detection is suppressed to the minimum. In addition,since the initial data 45A is automatically generated on the displayapparatus side, the user of the display apparatus is not requested toperform some operation. Therefore, the burden on the user is lessened.

In the embodiment, in the case where default data is pre-stored as theinitial data 45A in the storage 45, or in the case where an image fordetermination generated in the past on the basis of outputs (thelight-on light reception image and the light-off light reception image)of the light reception signal holder 33 in the past frame period T forexample is already stored as the initial data 45A by the process of thedetermination processor 51, a synthesis image obtained by combining theimage already stored as the initial data 45A and the image fordetermination may be overwritten as the latest initial data 45A in thestorage 45. In this case, when whether or not an image of an object tobe detected is included in the image for determination is determined onthe basis of a simple rule, even if it is erroneously detected that theimage of the object to be detected is not included in the image fordetermination in spite of the fact that an image of an object to bedetected is included in the image for determination, the influence ofthe erroneous operation is prevented from being prolonged. Further,deterioration in the precision of the position detection caused byerroneous detection is minimized.

In the embodiment, it is unnecessary to provide a part such as a touchpanel in order to detect the position of an object to be detected, sothat the configuration of the display apparatus can be simplified.

In the embodiment, in the case of intermittently driving the lightreception cells 13 in plural frame cycles, power consumption required todetect the position and/or the like of an object to be detected issuppressed.

Although the present invention has been described by the embodiment andits modification, the invention is not limited to the embodiment and thelike but can be variously modified.

For example, in the embodiment, each of the pixels 11 has the lightemission cells made by the red light emission cell 12R, the green lightemission cell 12G, and the blue light emission cell 12B. The pixel 11may also have a cell for emitting another color, or may have at leastone of the light emission cells.

In the foregoing embodiment, the case where the light reception cell 13detects visible light components in reflection light from the object tobe detected of visible light emitted from the light emission cell 12 andthe external light L2 has been described. Alternatively, the lightemission cell 12 may emit light including an invisible light component,and the light reception cell 13 may detect the invisible lightcomponent. Since the invisible light is invisible to human eyes,detection of an object which does not exert an influence on a displayimage is performed. Accordingly, no influence is exerted on the displayeven if the invisible light is caused to pass to the front surface side,in the case where the amount of transmission light of visible lighttraveling from the rear surface to the front surface is almost zero likein black screen display. Therefore, the object to be detected isdetectable also in the black display. More concretely, the light source100 shown in FIG. 3 is constructed to emit infrared light together withvisible light, and an infrared light transmission filter is disposed inan incident path of external light from the glass substrate 102B side tothe light receiving elements 16 shown in FIG. 2, and the light receivingelements 16 detect the infrared light component from the light source100 and an infrared light component in the external light as detectionlight. The absorption of the infrared light by a member such as thecolor filter 107 and the polarizer 101 is extremely lower than that ofvisible light. Therefore, the luminance of infrared light for obtaininga predetermined detection signal by the light receiving element 16 ismade lower than that of visible light, so that power consumption isreduced.

In the foregoing embodiment, the case where the display unit 1 has theliquid crystal display panel (the panel unit 110) on the light source100 has been described. The display unit 1 may have a light-emittingpanel in which a display element itself as a component of a pixel emitslight like an organic EL panel having an organic layer betweentransparent substrates facing each other. In this case, by turningon/off the display elements, display or no-display of an image, andturn-on or turn-off of the light are simultaneously performed.Consequently, it is unnecessary to provide the light source 100 at theback of the display unit 1 and simultaneously perform the operation ofturning on/off the display element and the operation of turning on/offthe light source 100.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2008-059501 filedin the Japan Patent Office on Mar. 10, 2008, the entire content of whichis hereby incorporated by reference.

Obviously many modifications and variations of the present invention arepossible in the light of the above teachings. It is therefore to beunderstood that within the scope of the appended claims the inventionmay be practiced otherwise than as specifically described.

1-13. (canceled)
 14. A position detecting method for detecting at leasta position of an object by a display apparatus including a panel parthaving a plurality of light emission cells which display an image on adisplay surface and emit detection light, and a plurality of lightreception cells which receive the detection light incident from a sideof the display surface; the method comprising: generating an image fordetermination on the basis of a first light reception image and a secondlight reception image, the first light reception image being obtainedfrom the plurality of light reception cells in a first period in whichthe detection light is emitted from the display surface, and the secondlight reception image being obtained from the plurality of lightreception cells in a second period in which the detection light is notemitted from the display surface; and determining whether or not animage of an object to be detected is included in the image fordetermination, wherein the first period is a display period in which theimage is displayed on the display surface, the second period is anon-display period in which the image is not displayed on the displaysurface, and the detection light is display light displaying the image.15. A position detecting method for detecting at least a position of anobject by a display apparatus including a panel part having a pluralityof light emission cells which display an image on a display surface andemit detection light, and a plurality of light reception cells whichreceive the detection light incident from a side of the display surface;the method comprising: generating an image for determination on thebasis of a first light reception image and a second light receptionimage, the first light reception image being obtained from the pluralityof light reception cells in a first period in which the detection lightis emitted from the display surface, and the second light receptionimage being obtained from the plurality of light reception cells in asecond period in which the detection light is not emitted from thedisplay surface; and determining whether or not an image of an object tobe detected is included in the image for determination, wherein theplurality of light emission cells emit invisible light as the detectionlight from the display surface, together with visible light displayingthe image on the display surface.
 16. The display apparatus according toclaim 14, wherein the image generating unit generates the image fordetermination by obtaining a difference between each pixel data of thefirst light reception image and each pixel data of the second lightreception image.
 17. The display apparatus according to claim 14,wherein the image generating unit generates a first reversed image and amoving-averaged image from the second light reception image, generates asecond reversed image and a predetermined threshold from themoving-averaged image, generates a difference image from a differencebetween the first reversed image and the second reversed image, andgenerates the image for determination by subtracting the threshold fromeach pixel data of the difference image.
 18. The display apparatusaccording to claim 14, wherein the image generating unit comprises: adifference image generating unit generating a first difference imagefrom a difference between each pixel data of the first light receptionimage and each pixel data of the second light reception image; a shadeimage generating unit for generating a first reversed image and amoving-averaged image from the second light reception image, generatinga second reversed image and a predetermined threshold from themoving-averaged image, generating a second difference image from adifference between the first reversed image and the second reversedimage, and generating a shade image by subtracting the threshold fromeach pixel data of the second difference image; and a synthesis imagegenerating unit for generating the image for determination by combiningthe first difference image and the shade image in accordance with apredetermined rule.
 19. The display apparatus according to claim 14,wherein the image determining unit detects whether or not a pixel regionhaving a value exceeding a predetermined threshold exists in each pixeldata of the image for determination and, when the pixel region isdetected, compares a size of the pixel region with a size expected asthe image of the object to be detected, to determine whether or not theimage of the object to be detected is included in the image fordetermination.
 20. The display apparatus according to claim 14, wherein,when the image of the object to be detected is determined to be notincluded in the image for determination and the image represented by theinitial data is already stored in the storage, the image determiningunit overwrites the storage with overwrite data as the initial data, theoverwrite data representing; an image for determination itself, an imageobtained by performing a predetermined process on the image fordetermination, or a synthetic image obtained by combining the imagealready stored in the storage as the initial data with the image fordetermination in accordance with a predetermined rule.
 21. The displayapparatus according to claim 14, wherein the position determining unitdetermines the position of the object to be detected from a differencebetween the image represented by the detection data stored in thestorage and the image represented by the initial data stored in thestorage.
 22. The display apparatus according to claim 14, wherein thefirst period is a display period in which the image is displayed on thedisplay surface, the second period is a non-display period in which theimage is not displayed on the display surface, and the detection lightis display light displaying the image.
 23. The display apparatusaccording to claim 14, wherein the plurality of light emission cellsemit invisible light as the detection light from the display surface,together with visible light displaying the image on the display surface.24. The display apparatus according to claim 23, wherein the panel partis a liquid crystal display panel having a liquid crystal layer betweentransparent substrates facing each other, and the invisible light isinfrared light.
 25. The display apparatus according to claim 14, whereinthe panel part is a liquid crystal display panel having a liquid crystallayer between transparent substrates facing each other.
 26. The displayapparatus according to claim 14, wherein the panel part is an organiclight emission panel having an organic layer between transparentsubstrates facing each other.